Minimax Regret Treatment Choice with Limited Validity of Experiments or with Covariates

This paper continues the investigation of minimax regret treatment choice initiated by Manski (2004). Consider a decision maker who must assign treatment to future subjects after observing outcomes experienced in a finite sample. A certain scoring rule is known to achieve minimax regret in numerous variants of this decision problem. I investigate the sensitivity of these findings to perturbations of the decision environment in realistic directions. They are: (i) The experiment may have limited validity, either because of selective noncompliance and realted problems or because the sampling universe is a potentially selective (in one of several ways) subset of the treatment population, thus samples generate potentially misleading signals even in the limit. These problems are formalized via a “bounds” approach that turns the problem into one of partial identification. (ii) Treatment outcomes may be influenced by a covariate whose effect on outcome distributions is bounded (in one of numerous probability metrics). This is interesting because introduction of a covariate with unrestricted effects leads to a pathological result. In both scenarios, small but positive perturbations leave the minimax regret decision rule unchanged (with caveats in one case). Thus, minimax regret analysis is not knife-edge dependent on ignoring certain aspects of realistic decision problems. Indeed, it recommends to entirely disregard covariates whose effect is believed to be positive but small, as well as small enough amounts of missing data. All findings are finite sample results derived by game theoretic analysis.